Serving deep learning models with deduplication from relational databases

Serving deep learning models from relational databases brings significant benefits. First, features extracted do not need to be transferred any decoupled systems for inferences, and thus the system management overhead can significantly reduced. Second, in a database, data along storage hierarchy is fully integrated with query processing, it continue model serving even if working set size exceeds available memory. Applying deduplication greatly reduce space, memory footprint, cache misses, inference latency. However, existing techniques are applicable applications databases. They consider impacts on accuracy as well inconsistency between tensor blocks database pages. This work proposed synergistic optimization duplication detection, page packing, caching, enhance serving. Evaluation results show that our improved efficiency latency, outperformed frameworks targeting scenarios.